{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fashion Mnist数据集多分类任务"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fashion MNIST数据集简介"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Fashion MNIST是一个替代MNIST手写数字集的图像数据集。它是由 Zalando（一家德国的时尚科技公司）旗下的研究部门提供，其涵盖了来自10种类别的共7万个不同商品的正面图片。\n",
    "\n",
    "FashionMNIST的大小（28x28）、格式（灰度图片）和训练集/测试集划分（60000/10000）与原始的MNIST完全一致。\n",
    "\n",
    "这个数据集的样子大致如下（每个类别占三行）：\n",
    "\n",
    "![image.png](attachment:image.png)\n",
    "\n",
    "数据集共有10个类别：\n",
    "\n",
    "  0: T-shirt/top（T恤）\n",
    "  \n",
    "  1: Trouser（裤子）\n",
    "  \n",
    "  2: Pullover（套衫）\n",
    "  \n",
    "  3: Dress（裙子）\n",
    "  \n",
    "  4: Coat（外套）\n",
    "  \n",
    "  5: Sandal（凉鞋）\n",
    "  \n",
    "  6: Shirt（汗衫）\n",
    "  \n",
    "  7: Sneaker（运动鞋）\n",
    "  \n",
    "  8: Bag（包）\n",
    "  \n",
    "  9: Ankle boot（踝靴）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 导入必要的模块"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "from tensorflow import keras\n",
    "from tensorflow.keras import datasets\n",
    "from tensorflow.keras import Input, Model\n",
    "from tensorflow.keras.layers import Flatten, Dense\n",
    "from tensorflow.keras.callbacks import EarlyStopping\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import time\n",
    "import os"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 载入Fashion MNIST数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(60000, 28, 28)\n",
      "(60000,)\n",
      "(10000, 28, 28)\n",
      "(10000,)\n"
     ]
    }
   ],
   "source": [
    "(x_train, y_train), (x_test, y_test) = datasets.fashion_mnist.load_data()\n",
    "print(x_train.shape)\n",
    "print(y_train.shape)\n",
    "print(x_test.shape)\n",
    "print(y_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "维度(60000, 28, 28) —> (60000, 784)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_train = x_train.reshape(-1, 28*28)\n",
    "x_test = x_test.reshape(-1, 28*28)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "归一化处理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_train = x_train/255.0\n",
    "x_test = x_test/255.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 模型搭建"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用到的api：\n",
    "\n",
    "**全连接操作`tf.keras.layers.Dense`**\n",
    "\n",
    "用到的参数：\n",
    "\n",
    "- **units**：输入整数，全连接层神经元个数。\n",
    "\n",
    "\n",
    "- **activation**：激活函数。\n",
    "\n",
    "  可选项：\n",
    "  \n",
    "  - 'sigmoid'：sigmoid激活函数\n",
    "  \n",
    "  - 'tanh'：tanh激活函数\n",
    "  \n",
    "  - 'relu'：relu激活函数\n",
    "  \n",
    "  - 'elu'或tf.keras.activations.elu(alpha=1.0)：elu激活函数\n",
    "  \n",
    "  - 'selu'：selu激活函数\n",
    "  \n",
    "  - 'swish': swish激活函数(tf2.2版本以上才有)\n",
    "  \n",
    "  - 'softmax': softmax函数\n",
    "\n",
    "\n",
    "- **kernel_initializer**：权重初始化，默认是'glorot_uniform'（即Xavier均匀初始化）。\n",
    "\n",
    "  可选项：\n",
    "  \n",
    "  - 'RandomNormal'或tf.keras.initializers.TruncatedNormal(mean=0.0, stddev=0.05)：正态分布采样，均值为0，标准差0.05\n",
    "  \n",
    "  - 'glorot_normal'：正态分布采样，均值为0，方差为2 / (fan_in + fan_out)\n",
    "  \n",
    "  - 'glorot_uniform'：均匀分布采样，范围[-limit, limit]，limit = sqrt(6 / (fan_in + fan_out))\n",
    "\n",
    "  - 'lecun_normal'：正态分布采样，均值为0，方差为1 / fan_in\n",
    "  \n",
    "  - 'lecun_uniform'：均匀分布采样，范围[-limit, limit]，limit = sqrt(3 / fan_in)\n",
    "  \n",
    "  - 'he_normal'：正态分布采样，均值为0，方差为2 / fan_in\n",
    "  \n",
    "  - 'he_uniform'：均匀分布采样，范围[-limit, limit]，limit = sqrt(6 / fan_in)\n",
    "  \n",
    "  fan_in是输入的神经元个数，fan_out是输出的神经元个数。\n",
    "\n",
    "\n",
    "- **name**：输入字符串，给该层设置一个名称。\n",
    "\n",
    "**模型设置`tf.keras.Sequential.compile`**\n",
    "\n",
    "用到的参数：\n",
    "\n",
    "- **loss**：损失函数，对于分类任务，如果标签没做one hot编码，一般使用\"sparse_categorical_crossentropy\"，否则使用\"categorical_crossentropy\"。\n",
    "\n",
    "\n",
    "- **optimizer**：优化器，这里选用\"sgd\"，更多优化器请查看https://tensorflow.google.cn/api_docs/python/tf/keras/optimizers\n",
    "\n",
    "\n",
    "- **metrics**：评价指标，这里选用\"accuracy\"，更多优化器请查看https://tensorflow.google.cn/api_docs/python/tf/keras/metrics\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "按照任务一的要求补全隐层的代码"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 输入层inputx\n",
    "inputs = Input(shape=(28*28), name='input')\n",
    "\n",
    "# 隐层dense\n",
    "x = Dense(units=256,activation='tanh',kernel_initializer='glorot_normal', name='dense_0')(inputs)\n",
    "x = Dense(units=128,activation='tanh',kernel_initializer='glorot_normal', name='dense_1')(x)\n",
    "\n",
    "# 输出层\n",
    "outputs = Dense(units=10, activation='softmax', name='logit')(x)\n",
    "\n",
    "# 设置模型的inputs和outputsin\n",
    "model = Model(inputs=inputs, outputs=outputs)\n",
    "\n",
    "# 设置损失函数loss、优化器optimizer、评价标准metrics\n",
    "model.compile(loss='sparse_categorical_crossentropy',\n",
    "              optimizer=\"sgd\", metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "查看模型每层的参数量和输出的大小"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "input (InputLayer)           [(None, 784)]             0         \n",
      "_________________________________________________________________\n",
      "dense_0 (Dense)              (None, 256)               200960    \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 128)               32896     \n",
      "_________________________________________________________________\n",
      "logit (Dense)                (None, 10)                1290      \n",
      "=================================================================\n",
      "Total params: 235,146\n",
      "Trainable params: 235,146\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 模型训练\n",
    "\n",
    "**`tf.keras.Sequential.fit`**\n",
    "\n",
    "用到的参数：\n",
    "\n",
    "- **x**：输入数据。\n",
    "\n",
    "\n",
    "- **y**：输入标签。\n",
    "\n",
    "\n",
    "- **batch_size**：一次梯度更新使用的数据量。\n",
    "\n",
    "\n",
    "- **epochs**：数据集跑多少轮模型训练，一轮表示整个数据集训练一次。\n",
    "\n",
    "\n",
    "- **validation_split**：验证集占总数据量的比例，取值0~1。\n",
    "\n",
    "\n",
    "- **shuffle**：每轮训练是否打乱数据顺序，默认True。\n",
    "\n",
    "返回：History对象，History.history属性会记录每一轮训练集和验证集的损失函数值和评价指标。。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 48000 samples, validate on 12000 samples\n",
      "Epoch 1/150\n",
      "48000/48000 [==============================] - 3s 58us/sample - loss: 0.6920 - accuracy: 0.7694 - val_loss: 0.5164 - val_accuracy: 0.8193\n",
      "Epoch 2/150\n",
      "48000/48000 [==============================] - 2s 47us/sample - loss: 0.4815 - accuracy: 0.8317 - val_loss: 0.4603 - val_accuracy: 0.8354\n",
      "Epoch 3/150\n",
      "48000/48000 [==============================] - 2s 46us/sample - loss: 0.4405 - accuracy: 0.8443 - val_loss: 0.4454 - val_accuracy: 0.8430\n",
      "Epoch 4/150\n",
      "48000/48000 [==============================] - 2s 46us/sample - loss: 0.4180 - accuracy: 0.8529 - val_loss: 0.4165 - val_accuracy: 0.8516\n",
      "Epoch 5/150\n",
      "48000/48000 [==============================] - 2s 49us/sample - loss: 0.4007 - accuracy: 0.8573 - val_loss: 0.4023 - val_accuracy: 0.8575\n",
      "Epoch 6/150\n",
      "48000/48000 [==============================] - 2s 47us/sample - loss: 0.3884 - accuracy: 0.8619 - val_loss: 0.4024 - val_accuracy: 0.8557\n",
      "Epoch 7/150\n",
      "48000/48000 [==============================] - 2s 48us/sample - loss: 0.3777 - accuracy: 0.8659 - val_loss: 0.3882 - val_accuracy: 0.8613\n",
      "Epoch 8/150\n",
      "48000/48000 [==============================] - 2s 47us/sample - loss: 0.3688 - accuracy: 0.8694 - val_loss: 0.3827 - val_accuracy: 0.8618\n",
      "Epoch 9/150\n",
      "48000/48000 [==============================] - 2s 47us/sample - loss: 0.3604 - accuracy: 0.8718 - val_loss: 0.3990 - val_accuracy: 0.8520\n",
      "Epoch 10/150\n",
      "48000/48000 [==============================] - 2s 49us/sample - loss: 0.3528 - accuracy: 0.8741 - val_loss: 0.3760 - val_accuracy: 0.8623\n",
      "Epoch 11/150\n",
      "48000/48000 [==============================] - 2s 48us/sample - loss: 0.3465 - accuracy: 0.8763 - val_loss: 0.3673 - val_accuracy: 0.8660\n",
      "Epoch 12/150\n",
      "48000/48000 [==============================] - 2s 48us/sample - loss: 0.3402 - accuracy: 0.8782 - val_loss: 0.3668 - val_accuracy: 0.8670\n",
      "Epoch 13/150\n",
      "48000/48000 [==============================] - 2s 48us/sample - loss: 0.3342 - accuracy: 0.8806 - val_loss: 0.3573 - val_accuracy: 0.8712\n",
      "Epoch 14/150\n",
      "48000/48000 [==============================] - 2s 52us/sample - loss: 0.3289 - accuracy: 0.8813 - val_loss: 0.3616 - val_accuracy: 0.8672\n",
      "Epoch 15/150\n",
      "48000/48000 [==============================] - 2s 49us/sample - loss: 0.3247 - accuracy: 0.8838 - val_loss: 0.3470 - val_accuracy: 0.8738\n",
      "Epoch 16/150\n",
      "48000/48000 [==============================] - 2s 48us/sample - loss: 0.3183 - accuracy: 0.8861 - val_loss: 0.3511 - val_accuracy: 0.8744\n",
      "Epoch 17/150\n",
      "48000/48000 [==============================] - 2s 52us/sample - loss: 0.3140 - accuracy: 0.8872 - val_loss: 0.3540 - val_accuracy: 0.8698\n",
      "Epoch 18/150\n",
      "48000/48000 [==============================] - 2s 46us/sample - loss: 0.3098 - accuracy: 0.8892 - val_loss: 0.3484 - val_accuracy: 0.8727\n",
      "Epoch 19/150\n",
      "48000/48000 [==============================] - 2s 46us/sample - loss: 0.3055 - accuracy: 0.8900 - val_loss: 0.3448 - val_accuracy: 0.8764\n",
      "Epoch 20/150\n",
      "48000/48000 [==============================] - 2s 46us/sample - loss: 0.3004 - accuracy: 0.8923 - val_loss: 0.3356 - val_accuracy: 0.8784\n",
      "Epoch 21/150\n",
      "48000/48000 [==============================] - 2s 47us/sample - loss: 0.2973 - accuracy: 0.8928 - val_loss: 0.3356 - val_accuracy: 0.8768\n",
      "Epoch 22/150\n",
      "48000/48000 [==============================] - 2s 46us/sample - loss: 0.2933 - accuracy: 0.8934 - val_loss: 0.3306 - val_accuracy: 0.8784\n",
      "Epoch 23/150\n",
      "48000/48000 [==============================] - 2s 46us/sample - loss: 0.2899 - accuracy: 0.8951 - val_loss: 0.3352 - val_accuracy: 0.8765\n",
      "Epoch 24/150\n",
      "48000/48000 [==============================] - 2s 46us/sample - loss: 0.2851 - accuracy: 0.8983 - val_loss: 0.3412 - val_accuracy: 0.8747\n",
      "Epoch 25/150\n",
      "48000/48000 [==============================] - 2s 46us/sample - loss: 0.2825 - accuracy: 0.8990 - val_loss: 0.3368 - val_accuracy: 0.8783\n",
      "Epoch 26/150\n",
      "48000/48000 [==============================] - 2s 46us/sample - loss: 0.2790 - accuracy: 0.8998 - val_loss: 0.3318 - val_accuracy: 0.8773\n",
      "Epoch 27/150\n",
      "48000/48000 [==============================] - 2s 47us/sample - loss: 0.2753 - accuracy: 0.9003 - val_loss: 0.3290 - val_accuracy: 0.8796\n",
      "Epoch 28/150\n",
      "48000/48000 [==============================] - 2s 47us/sample - loss: 0.2724 - accuracy: 0.9011 - val_loss: 0.3192 - val_accuracy: 0.8836\n",
      "Epoch 29/150\n",
      "48000/48000 [==============================] - 2s 46us/sample - loss: 0.2688 - accuracy: 0.9043 - val_loss: 0.3224 - val_accuracy: 0.8822\n",
      "Epoch 30/150\n",
      "48000/48000 [==============================] - 2s 45us/sample - loss: 0.2659 - accuracy: 0.9041 - val_loss: 0.3165 - val_accuracy: 0.8844\n",
      "Epoch 31/150\n",
      "48000/48000 [==============================] - 3s 58us/sample - loss: 0.2640 - accuracy: 0.9038 - val_loss: 0.3331 - val_accuracy: 0.8802\n",
      "Epoch 32/150\n",
      "48000/48000 [==============================] - 2s 51us/sample - loss: 0.2602 - accuracy: 0.9061 - val_loss: 0.3251 - val_accuracy: 0.8840\n",
      "Epoch 33/150\n",
      "48000/48000 [==============================] - 2s 50us/sample - loss: 0.2579 - accuracy: 0.9071 - val_loss: 0.3160 - val_accuracy: 0.8857\n",
      "Epoch 34/150\n",
      "48000/48000 [==============================] - 2s 46us/sample - loss: 0.2537 - accuracy: 0.9082 - val_loss: 0.3192 - val_accuracy: 0.8823\n",
      "Epoch 35/150\n",
      "48000/48000 [==============================] - 2s 47us/sample - loss: 0.2516 - accuracy: 0.9092 - val_loss: 0.3131 - val_accuracy: 0.8861\n",
      "Epoch 36/150\n",
      "48000/48000 [==============================] - 2s 47us/sample - loss: 0.2492 - accuracy: 0.9103 - val_loss: 0.3106 - val_accuracy: 0.8876\n",
      "Epoch 37/150\n",
      "48000/48000 [==============================] - 2s 46us/sample - loss: 0.2464 - accuracy: 0.9111 - val_loss: 0.3132 - val_accuracy: 0.8873\n",
      "Epoch 38/150\n",
      "48000/48000 [==============================] - 2s 47us/sample - loss: 0.2438 - accuracy: 0.9120 - val_loss: 0.3145 - val_accuracy: 0.8870\n",
      "Epoch 39/150\n",
      "48000/48000 [==============================] - 2s 46us/sample - loss: 0.2416 - accuracy: 0.9125 - val_loss: 0.3136 - val_accuracy: 0.8871\n",
      "Epoch 40/150\n",
      "48000/48000 [==============================] - 2s 46us/sample - loss: 0.2392 - accuracy: 0.9143 - val_loss: 0.3137 - val_accuracy: 0.8878\n",
      "Epoch 41/150\n",
      "48000/48000 [==============================] - 2s 48us/sample - loss: 0.2373 - accuracy: 0.9144 - val_loss: 0.3286 - val_accuracy: 0.8781\n",
      "Epoch 42/150\n",
      "48000/48000 [==============================] - 2s 46us/sample - loss: 0.2348 - accuracy: 0.9152 - val_loss: 0.3145 - val_accuracy: 0.8852\n",
      "Epoch 43/150\n",
      "48000/48000 [==============================] - 2s 50us/sample - loss: 0.2324 - accuracy: 0.9160 - val_loss: 0.3076 - val_accuracy: 0.8883\n",
      "Epoch 44/150\n",
      "48000/48000 [==============================] - 2s 46us/sample - loss: 0.2300 - accuracy: 0.9169 - val_loss: 0.3196 - val_accuracy: 0.8850\n",
      "Epoch 45/150\n",
      "48000/48000 [==============================] - 2s 48us/sample - loss: 0.2276 - accuracy: 0.9172 - val_loss: 0.3049 - val_accuracy: 0.8881\n",
      "Epoch 46/150\n",
      "48000/48000 [==============================] - 2s 47us/sample - loss: 0.2253 - accuracy: 0.9189 - val_loss: 0.3103 - val_accuracy: 0.8868\n",
      "Epoch 47/150\n",
      "48000/48000 [==============================] - 3s 54us/sample - loss: 0.2223 - accuracy: 0.9203 - val_loss: 0.3141 - val_accuracy: 0.8886\n",
      "Epoch 48/150\n",
      "48000/48000 [==============================] - 2s 49us/sample - loss: 0.2210 - accuracy: 0.9208 - val_loss: 0.3019 - val_accuracy: 0.8926\n",
      "Epoch 49/150\n",
      "48000/48000 [==============================] - 2s 50us/sample - loss: 0.2180 - accuracy: 0.9224 - val_loss: 0.3112 - val_accuracy: 0.8868\n",
      "Epoch 50/150\n",
      "48000/48000 [==============================] - 2s 51us/sample - loss: 0.2155 - accuracy: 0.9233 - val_loss: 0.3192 - val_accuracy: 0.8863\n",
      "Epoch 51/150\n",
      "48000/48000 [==============================] - 2s 50us/sample - loss: 0.2137 - accuracy: 0.9241 - val_loss: 0.3103 - val_accuracy: 0.8912\n",
      "Epoch 52/150\n",
      "48000/48000 [==============================] - 2s 48us/sample - loss: 0.2118 - accuracy: 0.9243 - val_loss: 0.3116 - val_accuracy: 0.8870\n",
      "Epoch 53/150\n",
      "48000/48000 [==============================] - 2s 47us/sample - loss: 0.2099 - accuracy: 0.9252 - val_loss: 0.3082 - val_accuracy: 0.8892\n",
      "Epoch 54/150\n",
      "48000/48000 [==============================] - 2s 48us/sample - loss: 0.2078 - accuracy: 0.9251 - val_loss: 0.3079 - val_accuracy: 0.8908\n",
      "Epoch 55/150\n",
      "48000/48000 [==============================] - 2s 47us/sample - loss: 0.2057 - accuracy: 0.9259 - val_loss: 0.3110 - val_accuracy: 0.8894\n",
      "Epoch 56/150\n",
      "48000/48000 [==============================] - 2s 47us/sample - loss: 0.2041 - accuracy: 0.9274 - val_loss: 0.3035 - val_accuracy: 0.8913\n",
      "Epoch 57/150\n",
      "48000/48000 [==============================] - 2s 47us/sample - loss: 0.2023 - accuracy: 0.9270 - val_loss: 0.3001 - val_accuracy: 0.8918\n",
      "Epoch 58/150\n",
      "48000/48000 [==============================] - 2s 47us/sample - loss: 0.1994 - accuracy: 0.9289 - val_loss: 0.3063 - val_accuracy: 0.8916\n",
      "Epoch 59/150\n",
      "48000/48000 [==============================] - 2s 46us/sample - loss: 0.1984 - accuracy: 0.9286 - val_loss: 0.3046 - val_accuracy: 0.8911\n",
      "Epoch 60/150\n",
      "48000/48000 [==============================] - 2s 46us/sample - loss: 0.1962 - accuracy: 0.9298 - val_loss: 0.3037 - val_accuracy: 0.8930\n",
      "Epoch 61/150\n",
      "48000/48000 [==============================] - 2s 46us/sample - loss: 0.1939 - accuracy: 0.9305 - val_loss: 0.3027 - val_accuracy: 0.8909\n",
      "Epoch 62/150\n",
      "48000/48000 [==============================] - 2s 46us/sample - loss: 0.1918 - accuracy: 0.9310 - val_loss: 0.3074 - val_accuracy: 0.8899\n",
      "Epoch 63/150\n",
      "48000/48000 [==============================] - 2s 46us/sample - loss: 0.1900 - accuracy: 0.9325 - val_loss: 0.3107 - val_accuracy: 0.8885\n",
      "Epoch 64/150\n",
      "48000/48000 [==============================] - 2s 46us/sample - loss: 0.1885 - accuracy: 0.9328 - val_loss: 0.3076 - val_accuracy: 0.8917\n",
      "Epoch 65/150\n",
      "48000/48000 [==============================] - 2s 46us/sample - loss: 0.1859 - accuracy: 0.9343 - val_loss: 0.3055 - val_accuracy: 0.8920\n",
      "Epoch 66/150\n",
      "48000/48000 [==============================] - 2s 46us/sample - loss: 0.1845 - accuracy: 0.9348 - val_loss: 0.3143 - val_accuracy: 0.8903\n",
      "Epoch 67/150\n",
      "48000/48000 [==============================] - 2s 48us/sample - loss: 0.1829 - accuracy: 0.9360 - val_loss: 0.3074 - val_accuracy: 0.8907\n"
     ]
    }
   ],
   "source": [
    "earlystopping = EarlyStopping(monitor='val_loss',min_delta=1e-4,patience=10,restore_best_weights=True)\n",
    "history = model.fit(x=x_train, y=y_train, batch_size=32,\n",
    "                    epochs=150, validation_split=0.2,\n",
    "                    shuffle=True,callbacks=[earlystopping])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "画图查看history数据的变化趋势"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeMAAAE9CAYAAAA4QwpnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeXxU1fn48c+ZJZkkk30jYQvIEpYQdhAUAlTAFkUtKmBRqctXrdrKV+vSav3VLu5Wq98i7lYpbtW6IFqEQLGILAIBApGdAIHsyWSd5fz+uFmBwASTDJl53q/XvO6dO2fufU7QPDnLPVdprRFCCCGE75h8HYAQQggR6CQZCyGEED4myVgIIYTwMUnGQgghhI9JMhZCCCF8TJKxEEII4WMWX104Li5Op6SktNn5KioqCAsLa7PzdRaBWO9ArDMEZr0Dsc4QmPUOlDpv3LixQGsdf+JxnyXjlJQUNmzY0Gbny8zMJCMjo83O11kEYr0Dsc4QmPUOxDpDYNY7UOqslDpwquPSTS2EEEL4mCRjIYQQwsckGQshhBA+JslYCCGE8DFJxkIIIYSPSTIWQgghfEySsRBCCOFjkoyFEEIIH5NkLIQQQviYz1bgEkIIIXxOa3DVQHVp3auk+f6I+WAyt3sYkoyFEEJ0DlobSbKqCCqLwVUNHhd4nOBxG/tuJ9SUQUU+VBRCZUHdfgHUlBufu2vAVVu3rQF0y9ccdAWExrR71SQZCyGEODuuGiMhmixgsoLZCkoZn2kNzqqTW5w15VBbYXzmrGzY9j2wB0rea0yQ7lrj3K4aqCqGyiIjCXtc3sdnDYOwWAiLh/AkiO8P5mCwBBlbsxUswWANAVsU2CKbbCMhJMp43wEkGQshRKDQ2mgZOiuM5FZR12qsLDD2q4qNMuYmydVkMV7VpVCeB448Y1t+1Ch/ImU2vudxGy3WM1IQFEa8NoPDDuYgI0HWby02iOsLobEQEmNsQ2ON1qrF1hifyWJ0J5ssYIuA0DgICm3zH2F7kWQshBAdwe2C2rpWYW0F1DiMpOisOqGlWEmPA7sgcx1oN2iPkdi0x2gl1pYbrcuacuMcNeXgqgIUKJPRMlUm4z0anNXG5/Xn156WY7SGGt91O41E2rSsyQL2RAjvAtG9oMf5xr411CjrdjV2GbudRmI8saVpizISpTXUaI1aQ42EqxT/DZCnNrVEkrEQQpyJsxoqC41XdUnjeKO7tnG/oUxdK7OioHG/ptwo46XeAPvq3ykjsSmz0VoMDm/yskN4IlhCAG20arWncR/qkl594rM17ofGGl24oXEQFmdsrbbmgXg8jcnVGgomuQGnvUgyFkL4L7cTah1GC9JdW9dyczVO9vG4jZamI79uks9xI3k6jhuJtLLQ6M6tdXh/zZBoY4wyNA7i+hktSFsEBIVDUFiTl93oRrWG1W1DjH1rCKu+/oaJGZMaW7q+YjKBKdhovYp2JclYCOEbbpeR/MrzGscqm7bo0MQWbIZthVBrdN82686tdTQ53mS/prxJAva+NQoY46T2BKOlGBYPcf0bxydDY43jtiijhVk/+ccc1Di+GRJtHP+BdP34pwgYkoyFEGfm8TSMZxoJsbIu+VU0joHWOhr3nVV1Xbh1Xbn1+zXl4DhmJODKQk57SwmQBrDtxKPGhB+soY2tzPp9e0JjqzPY3tgaDbYbCVSZTp7sYw2tS8DxxrimL1uiImB5lYyVUtOBZwEz8LLW+tETPo8GXgXOA6qBn2utT/pfSAjRwbQ2kmN1qXHvZf2tJTVljZN/ah1NJgSVQXVZk/JlxtZZ2brrmoOa30JiqWs9Btkhqgd0Gwn2LsZ4p72L0fKsn3TUkAwVGzdtYsTYC5t351pskjCF3zljMlZKmYEXgIuAXGC9UupjrfWOJsUeADZrrS9XSqXWlZ/SHgELEdDcTijNhZKDja+K402SaZOkWlNuJFPtPvN5rXWtx+AIY3zTFgmR3Yz94Igm45t1LdD68c1m459N3rdRF2v57nJISG2TcwlxLvOmZTwa2K213guglFoCzASaJuOBwJ8BtNY7lVIpSqlErfWxtg5YiE7F44HSg3B8J+RnN24L9xhdt8p0wks1X0DBZKnbWhlbegxWFTW/3USZjLHM4IjGGbZRPRv3bZGNCbU+yQbXvQ+2G2WC7DI+KYSPeZOMuwKHmrzPBcacUGYLcAWwRik1GugJdAMkGYtzl9bGCj/1E3+aTRBqvjpQw7Z+RaBmS+rVrRTUMJbaZOy0utT4rF54EsSnwrDzjdal9tS9dOP9pA33aja5Z9PjokTF0yV1jNHNG9nd2EZ0NbqAhRCdmtL69BMolFJXAtO01jfWvZ8HjNZa39GkTATGmPIwIAtIBW7UWm854Vw3AzcDJCYmjliyZEmbVcThcGC329vsfJ1FINa71XXWmuCaAsIqDhJWcZDQyoOEVRwitPIQFnf1mb/f9FQoPCYrWlnxmCx4TJa6fStus63JK6RuG0plaDIVYT2oDO2Oy3r2/1bybx04ArHegVLnSZMmbdRajzzxuDct41yge5P33YAjTQtorcuA+QBKKYVxu/o+TqC1XgQsAhg5cqRuy9VWMgN09Ra/rbfWxrinq+k6tcZ20/qdDO85sMlC79XGvqvKWBjeccxYss9xvG7m7rG6FYrqhCUY45DxkyEiqW41oLr7PBvGQ0ObrxJUt1VmK2YfTR7y23/r0+gsddZuN67CQizx8ag2+O+js9TbW9rlwlVYiOv4cTwVlVgSErB2ScQU2rhcpb/VubW8Scbrgb5KqV7AYWA2MLdpAaVUFFCpta4FbgRW1yVoIQz1s3prK06+HaayGEoOQOkhKDlkTEoqzW2eQJsYDvDdaa4VEm0s22dPgG6jjW1Mb6N7OGFAhzyBRfg357HjVG3dQvXWLKq2bqV62zY8FRVYu3XDPnkS4ZOnEDpiOMrq/T3HrsJCKteto+Lbbwk/dozScgchw4Zi7dr1tAlea427pAR3YSGuoiLcRUW4CgtxFxXjcTgwRUZgjorCEhWFOSoKc3Q0Jrsdd3Exrvx843W8bltYiKeqEl1Ti66pQdfW4KnbB8BsQpnMKLMZzHVbqwVltWKyBqGCGl/a6cR17BjO/OO4Cwqb3D/eyBQZibVLFyxdEgl3ezj27XpM9jDMdjumsDBMYXZMdjvmyEjM0dGYo6IwhYU2/Dy0x4PzyBFqcr6nJieHmu+/p2b3btxlZeB2o93uZltls2GJizNe8XGY4+KwxMVjsoeha2vRtc66bS3aaezHL7gLU1D7DwWdMRlrrV1KqduBLzBubXpVa71dKXVL3ecLgQHAm0opN8bErhvaMWZxLvB4jJWL6m99abqtOA5lR4yF5MuO1O3nnXkBhtBYYyw0YQD0m2ase2ux1S2s0PiEla3bdzFk+Ki6ReTrPqtfUD40RlYLOkva5cJdUmL8Qi8uwV1cZPzCLjYW5FBmC8pqMX4JW6woi8X45Wy2oMwmaNiajYloHt18IQ+t0W43nupqdFU1nppqdHUNuqYak91O1FVXYYlp3R9KbkcFtXt2G7+Ev99Nzf59WKKiCe7Xj+D+/bH174c5Lq7xl7fWuPLzqd2zh5rde6jZsxtPRSXmiAjMkRGYIiIwR0RijowAs9lIcPkFuArqX/k4D+XiOlY3HcZiwZaaSuTMS7F2607lunWULHmH4jf/jikyEvuECdgzJmJNTGxMVFarsTWbqc7OpuKbdVR+8w01338PgCksjBCnkyMrMwEwx8URMjSd0KFDMcfG4TxyGOfRo7iOHMF5+AjOo0fRtbWn/PmokBB01an/qG1eUGGOicESG4vJbkcFB2EOD0cFBxuvoLqnMbncaI8b3J66BOdCO10NictTUYGnpBhdW4uyWLEkJmAbNBBLfAKWBONlCg3Bdfw4zqN5uI7l4Tyah/NYHsG5hynesAFdfYahI6sVc1Qk5vAIXHl5eCobb7uzJicT1LcPtoEDURYzNP3DwWTCU13d8O9Yu38/roKCFn92WCyooCDibrsVzoVkDKC1XgosPeHYwib7a4G+bRua8CmPG8oOQ/GBxltoSg817pcdPv2jzCw2Y7JSRFfoPtrYD42tW4jB3mTBBrvxmLLIbsZ7LxTlhUGvC9uoou3HU1uL8+BBTCEhmKOiUKGhJ7Vw6ls1zsNHcB45jCvvGJaEBEKGDcOamHD681dXU52dTe2BA0bSKCjEVViAu8BoIWmXk+Dz+hDcvx+2/v0J7tfPaGXV/VKq2bmTqu3bqd6+g+pt26jZswfcXtwG1caU1Yp2uShY9BIx184jdv58zJGRpyzrqa6m/N/LKfv8c6p3ZuM6crTxPDYbQSkp1OzIpvRf/2o4bo42krOuraVmzx48ZY2ddkbyjcBdXm4cb2EOjSkioqFFFTp6NCGDB2EbMgTbwIGYghv/+Iv9+Xw8lZU4vv4ax1crcGRmUvbJJ6evv81G6PDhRFxyCWFjx2AbOJBVq1YxNimJys2bqdq8marNW3As/6qxTvFxWJOTCR44APuUKVi7JGKOjcUSG4s5OgZLbIzx35zFgq6txV1aavyhVVyMu6QEj6PCaC3Hx2NJiMcSE9OqVnx7qO+m1k6nkdQrKnA7KvA4yo34i0twlxjxG69SwsaNI7hfX4L7Gi9zK8ectdZ4ysvxVFQ0a9UrqxXVwetwywpcgcLtNJJp+bHG2cGu6sZtTZmRZIsPQPF+o2yzZKuMhBrVA7qPMZJnWFzdLTQRjbfPBIcbKxmFRAfUwgxaa5wHD1K1dStVW7ZStXUrNdnZaGfjI+RUUJDRTRgVhSkiHHdxCc4jR1psuViTkwkZNoyQ4cMIHTYMy4EDFC95h6ptWVRv2260pJokT2WzGb+MY2OxJiWBUlRnZ1P+xRcNZUyhoVgSEqg9dKjhu+aYGGyDBmHPyMDSJRFLdDTm6BjM0dFYYoyuQcxmtMsFTifa7Ua7jBYRHjfa7TFaSA3bupiUwljEA+OPEKXAbMZks6FsNkz1rS6zmZq9eyl4/nkKF75I8duLif35fKLnXdvws63esoWSf35I2dKleBwOrMnJhA4fQfBVfQju24fgvn2NPzTMxi1aruJianblUJOTQ3XOLmq+/x5ltRLx44sJ7tOX4D7nEXzeec1bzR4PnvJy3GVluEvLwO3CEmd0ZTZNuGdiCg0l4qKLiLjoIrTbTfWObDzlZXiadX860c5agnv1wpaefnI3qNmMbeBAbAMHwlxjVNBVVISnrAxLUlKr4lFBQUbSjY+nM/QZKau14f+T9v7zQCll9IpERLTzlc5MkrE/qJ/sVP+M0fI8Y8y1eL/xKjlgvD/do9PAaLlGp0DyMBh0OUT3NO5Zje4JEd3a9Raa+gkwrmPHcR0/hjMvD0+5w/ifMjYGS0xMQzcaHk9jd2pREa7CIqNLtaQUc2REQ3eYJSEBS2ys0Z3qZQy1+/dTvSMbd2mpMWbVdPzKbgetcR45gvPw4SavI9Tu34+7tBQwugZDBg0iet48bKn9jZZJk1aJu6QUd2kJwb17Yb9gPNbkZCzJyViTkrF2ScR55AhV331H5XebqVy/nrLPPgMgFsgDzFFR2AYPxj4pg5DBg+uSSnyzsbSmPBUV1OzeTXVODjW7cnAdyyP84umEDBqEbdAgLF26eDXpSAUFtVt3XXDv3nR9+mlib76Z/Of+Sv6zz1H0xpuEDxvK3sefoHbvXlRICBFTpxJ5xRWEjhp52paLJToay9gxhI098S7MlimTyRibjIxsPmX1B1BmMyFpg9vkXJaYGGhlN77oPCQZdwa1FUYyLT1Ut801JjqV5jL6+B74uvTUyxXaE41k2n0sDOlpJNqI5OaPUqvf1q+cdAKtNTideKpr0TXl6JoaPDU1aKeToO7dMYWEtKoqWmuch49QvX17w6tm715c+fngOk23dxOJQI63FzSZsMTGNrQMLAnxDfvmuDjcxcVUZ2dTsyOb6pwc78bX6lmtWJOSsHZNJnzqVGxpgwkZMoTgPn28/gPgVCxxcYQMGULMddcZY5xHj1K1eTPbs7MZcfXVZ5zQcyJTWBgh6emEpKefdUwdxZaaSvf/e4GqrVvJf/Y53CszMQ8fTtIfHiF8+vRWd0MK0VlIMj4XVJXUjcUeqEuyh5qM0R6CqqLm5ZXZSKqR3SgP70PoeenGZKfwpCbbJGP5wia0y4WnoqLF8biGcrW1ONasofSTT3CszGx5QoXVSkhaGqGjRxE6ahShw4Y1u1XBVVxM7b591O7bR83evdTs3EX19u24S0qMAhYLwX37EjZ6FJZEY0alNTERS0KicdtDeLgxVlRYiKuoGHeRMRa6NyuL84YMwRwTizkmumGczBwVibu01JgZevx43esYzuPHceXn48w/TtWO7bgLi4wJaHVMdju21FSirpyFbcBAbAMHYImLaxi38jgcuB0OPBWVoDXWrslYu3bFEhfX0C3aXpRSWJOTsSYnUxMaSlC3bu16vXNFyJAh9HjlZTK//JKMqVN9HY4Q7U6ScUdxOyF/J+RlwbHtjd3HxQehprR5WWto3QpL3SF5uLGN7GGM00Z1NxbWNxv/dNmZmSS2cG+eq6iIqs1bjAkgW7ZQlZWFrqzEmpyMbcgQQoYMISTdmISigoOp2rSJ0k8+pXzZMtylpZijo4mcORNrUlLdrMogTEFBqOBgMJmo2bmTim+/pfCllylc+CJYLIQMHgxmM7V79+IuLm6IRVmtBPXpg/1HU4zu0cGDCe7X74xjXyabDWtiYrNjWZmZxLRQZ1NCAtaEBGBQi+fULheuoiJc+fmYw8Oxdut26i7P2NjTxiY6QAfMYhXiXCDJuD24aoyke3gT5G0x9o9nGwtXgNE1HJ1SNxlqrLGN7mlso3qecfKT9nio3beP6h07CP36v+Rv3VrXcqvA4zBacrW5h3AeOFh3PeP2i6grrsCSmED1jh1Ub9lK+bJlxudmM+bISNxFRaiQEMInTybikhnYx48//QzLn/wEMG4vqfruOyq//ZbKDRuMuV4/+hFBvXsT3LsXQb17Y01ObvdWpLeUxYK1IWkLIYTvSTL+obSGor1weCPkboDDG4zkW594Q2IgaQiMuQWdmIaOHQAxvTGFencbD0Btbi6VGzYYSXTHDmp2ZDfcWxcOFCiFKTQUk924Qd4UFoatXz+ir7ySkKFDsQ0adMqxXVdBAVVbs6jaugXn4SPYL7yA8ClTMIV5HxuA2R6G/cILsF94Qau+J4QQwiDJuLU8HuOpO/u/hgNfw4H/GotcAFhDcUcPwWG6mPJ9NVTtPoKuceJx5qKd+8D5vlFOKSJ+8hPi77idoJ49W7yUKz+f/L8+T8n774PHg7LZjAUGLr/cuO1h0EDW7dvHhGnTzuqeOEtcHOGTJxE+edLZ/CSEEEK0EUnGZ6BdtdSs/QzH0o9wHtiDxZ2HxVqBxebBEheLpd84PP0H49hbi2NDNpXfbQb3QcxxcYSdf76xis0JK++4Cgsoee99ypYtI+qnPyXutlubjYt6KispfO01Cl95FV1bS/TcuURffRVBvXuf1NWr8/I6/OZ0IYQQbUuS8YlqK9D7/0vVqs8o/883lGcX4Sw3AxqzTeGusYKOqiusgW/rXhDcvz+xN99E+KRJ2AYPPm2SjLvpJgoWvkjxu+9S+tFHRM+dS+wNP6d85UoKnvsrrvx8wqdOJWHBXQSlpLRzpYUQQviSJON6ZUep/exJit79iLJ9ZtzVZjBBWL8kYieOw37Zz7D2GmQsfl5U1Liwen4+gLF4Q9euXl/OEh9Plwd/S8z8+RQ8/zxFb7xB0euvg9aEDB1K12efJXT4sHaqrBBCiHOJJOPj2VS9+yeKPl5D2aFgMAUTPjad8Et+in3KNMzh4c2KK6sVa2LiSbfbnK2gbl1JfvTPxN54AyXvvU/I8OGET72oTR7DJoQQonMI2GSsD6yl4vXfU/jVLiqPB2Oy2YmZO5OYm+9os0TbGsF9+pB4/30dfl0hhBC+F3DJ2F2QR+lTt1GyYis1pVYsUdEk/Go+Uddcd1IrWAghhOgIAZOMq7K2UfLKs5QuX4N2ga1bPEl3/4rImZcbC+ALIYQQPuLXyVhrTemHH1H81t+p3pGNMnuI7Gcl6he/IeRHc3wdnhBCCAH4cTL21NRw9IHfUPbZZwTHKBJHlBD507mYZzxy0gMUhBBCCF/yy2TsKi4m9xe3U7VpE/HpDmLHRqEufw9SZLlGIYQQ5x6/S8Y1+/Zx6H9uwZWXR9fbfkxE0cswbwXE9fV1aEIIIcQp+dU6ihXffsv+2XPwOBz0eON1IqL3QmwfScRCCCHOaX6TjG3frOPgDTdiiY0l5Z0lhA7sA/vXQL/pvg5NCCGEOC2/SMblK1YQ+frrhI4YQco/FhPUvTvsWWE8xrD/xb4OTwghhDgtv0jG9gsuoPynP6XHohcxR0YaB3ctA1sUdB/r2+CEEEKIM/CLZKyCgqi86EeNi3d43PD9F9D3IjD73Rw1IYQQfsarZKyUmq6U2qWU2q2UOmkBZaVUpFLqE6XUFqXUdqXU/LYPtRVyN0BloYwXCyGE6BTOmIyVUmbgBeBiYCAwRyk18IRivwB2aK3TgQzgKaWU79aYzPkcTBbo8yOfhSCEEEJ4y5uW8Whgt9Z6r9a6FlgCzDyhjAbClfHcPztQBLjaNNLW2PU59BwHIVE+C0EIIYTwljfJuCtwqMn73LpjTT0PDACOAFnAL7XWnjaJsLWK9kH+Tugns6iFEEJ0Dt7MbjrVU+71Ce+nAZuBycB5wL+VUv/RWpc1O5FSNwM3AyQmJpKZmdnqgFvicDjIzMyka+4n9AW+KYmmug3Pf66qr3cgCcQ6Q2DWOxDrDIFZ70Csc1PeJONcoHuT990wWsBNzQce1VprYLdSah+QCnzbtJDWehGwCGDkyJE6IyPjLMM+WWZmJhkZGfDG0xDXn7EXB8ZTmRrqHUACsc4QmPUOxDpDYNY7EOvclDfd1OuBvkqpXnWTsmYDH59Q5iAwBUAplQj0B/a2ZaBeqS6FA19Df5lFLYQQovM4Y8tYa+1SSt0OfAGYgVe11tuVUrfUfb4QeAR4XSmVhdGtfa/WuqAd4z613V+BxyXjxUIIIToVr1bE0FovBZaecGxhk/0jwNS2De0s5CyDkBjoPtrXkQghhBBe84sVuACUxw3ffwl9p4LJ7OtwhBBCCK/5TTKOKNsJVcUyXiyEEKLT8ZtkHFu4HkxWOG+Kr0MRQgghWsW/knHKeLBF+DoUIYQQolX8IxkX7iGsMldmUQshhOiU/CMZVxZSbu8t48VCCCE6Jf9Ixt1Hs3HkMxCd4utIhBBCiFbzj2QshBBCdGKSjIUQQggfk2QshBBC+JgkYyGEEMLHJBkLIYQQPibJWAghhPAxScZCCCGEj0kyFkIIIXxMkrEQQgjhY5KMhRBCCB+TZCyEEEL4mCRjIYQQwsckGQshhBA+JslYCCGE8DFJxkIIIYSPSTIWQgghfEySsRBCCOFjXiVjpdR0pdQupdRupdR9p/j8HqXU5rrXNqWUWykV0/bhCiGEEP7njMlYKWUGXgAuBgYCc5RSA5uW0Vo/obUeqrUeCtwPrNJaF7VHwEIIIYS/8aZlPBrYrbXeq7WuBZYAM09Tfg7wj7YITgghhAgE3iTjrsChJu9z646dRCkVCkwHPvjhoQkhhBCBQWmtT19AqSuBaVrrG+vezwNGa63vOEXZq4Gfaa0vaeFcNwM3AyQmJo5YsmTJDwy/kcPhwG63t9n5OotArHcg1hkCs96BWGcIzHoHSp0nTZq0UWs98sTjFi++mwt0b/K+G3CkhbKzOU0XtdZ6EbAIYOTIkTojI8OLy5/Z8fJqXv5kDbdMGkdMWFCbnLOzyMzMpK1+jp1FINYZArPegVhnCMx6B2Kdm/Kmm3o90Fcp1UspFYSRcD8+sZBSKhKYCPyrbUM8s93HHCzaWsPOvLKOvrQQQgjxg52xZay1dimlbge+AMzAq1rr7UqpW+o+X1hX9HLgS611RbtF24K48GAACh21HX1pIYQQ4gfzppsarfVSYOkJxxae8P514PW2Cqw1Yuu6pgscNb64vBBCCPGD+MUKXNGhQZiUJGMhhBCdk18kY5NJER6kKCiXbmohhBCdj18kY4CIICUtYyGEEJ2S3yTjyCBFQYW0jIUQQnQ+fpOMw4OhoFxaxkIIITofv0nGkXXd1GdaUUwIIYQ41/hNMo4IVtS4PDhqXL4ORQghhGgVv0nGkUEKkIU/hBBCdD5eLfrRGUTUJeMCRw0pcWE+jkYIITqW0+kkNzeX6upqX4dyViIjI8nOzvZ1GG3GZrPRrVs3rFarV+X9JxkHNyZjIYQINLm5uYSHh5OSkoJSytfhtFp5eTnh4eG+DqNNaK0pLCwkNzeXXr16efUdv+mmrm8Z50s3tRAiAFVXVxMbG9spE7G/UUoRGxvbql4Kv0nG4fXd1HJ7kxAiQEkiPne09t/Cb5KxxaSIDrVSWCHJWAghROfiN8kYIM4eLOtTCyGEj9jtdl+H0Gn5VTKOtQfJBC4hhBCdjl8l4zh7sCRjIYTwMa0199xzD4MHDyYtLY133nkHgKNHjzJhwgSGDh3K4MGD+c9//oPb7eb6669nzJgxpKWl8cwzz/g4et/wm1ubwEjGsuiHECLQ/b9PtrPjSFmbnnNgcgS/u2SQV2X/+c9/snnzZrZs2UJBQQGjRo1iwoQJLF68mGnTpvGb3/wGt9tNZWUlmzdv5vDhw6xbt47w8HBKSkraNO7Owq9axvHhwZTXuKh2un0dihBCBKw1a9YwZ84czGYziYmJTJw4kfXr1zNq1Chee+01Hn74YbKysggPD6d3797s3buXu+++m2XLlhEREeHr8H3Cz1rGQYCx8Ee36FAfRyOEEL7hbQu2vbT0wJ4JEyawevVqPvvsM+bNm8c999zDtddey5YtW/joo4944YUXePfdd3n11Vc7OGLf86uWcWxYMAAF0lUthBA+M2HCBN555x3cbjf5+fmsXr2a0aNHc+DAARISErjpppu44YYb2LRpEwUFBXg8HmbOnMkjjzzCpk2bfB2+T/hXyzi8LhnLwh9CCOEzl19+OWvXriU9PR2lFI8//jhdunThjTfe4M4Yo/IAACAASURBVIknnsBqtWK323nzzTc5fPgw8+fPx+VyYTKZ+POf/+zr8H3Cv5JxXTe1LPwhhBAdz+FwAMbqU0888QRPPPFEs8+vu+46rrvuupO+t2nTJr9am/ps+FU3dZxduqmFEEJ0Pn6VjG1WM+HBFvKlm1oIIUQn4lUyVkpNV0rtUkrtVkrd10KZDKXUZqXUdqXUqrYN03uyCpcQQojO5oxjxkopM/ACcBGQC6xXSn2std7RpEwU8H/AdK31QaVUQnsFfCayCpcQQojOxpuW8Whgt9Z6r9a6FlgCzDyhzFzgn1rrgwBa6+NtG6b3ZBUuIYQQnY03ybgrcKjJ+9y6Y031A6KVUplKqY1KqWvbKsDWiguXbmohhBCdize3Np3qCcknLq9iAUYAU4AQYK1S6hutdU6zEyl1M3AzQGJiIpmZma0OuCUOh4PMzEwcBbUUVzpZvmIlFpP/P2i7vt6BJBDrDIFZ70CsM5xdvSMjIykvL2+fgDqA2+3u1PGfSnV1tdf/jt4k41yge5P33YAjpyhToLWuACqUUquBdKBZMtZaLwIWAYwcOVJnZGR4FaQ3MjMzycjI4JDtAB/t3kbayPNJjLC12fnPVfX1DiSBWGcIzHoHYp3h7OqdnZ3dqe/T9fY+Y5fLhcXSOZbIsNlsDBs2zKuy3nRTrwf6KqV6KaWCgNnAxyeU+RdwoVLKopQKBcYA2a2Iuc3EN1mfWgghRMe57LLLGDFiBIMGDWLRokUALFu2jOHDh5Oens6UKVMAo+U/f/580tLSGDJkCB988AEAdru94Vzvv/8+119/PQDXX389CxYsYNKkSdx77718++23jBs3jmHDhjFu3Dh27doFGK3ru+++u+G8f/3rX/nqq6+4/PLLG87773//myuuuKIjfhytcsY/L7TWLqXU7cAXgBl4VWu9XSl1S93nC7XW2UqpZcBWwAO8rLXe1p6Bt0QW/hBCBLzP74O8rLY9Z5c0uPjR0xZ59dVXiYmJoaqqilGjRjFz5kxuuukmVq9eTa9evSgqKgLgkUceITIykqwsI8bi4uIzXj4nJ4fly5djNpspKytj9erVWCwWli9fzgMPPMAHH3zAokWL2LdvH9999x0Wi4WioiKio6P5xS9+QX5+PvHx8bz22mvMnz//h/882phXbX2t9VJg6QnHFp7w/gmg+dpnPtCQjGXhDyGE6FDPPfccH374IQCHDh1i0aJFTJgwgV69egEQExMDwPLly1myZEnD96Kjo884XnzllVdiNpsBKC0t5brrruP7779HKYXT6Ww47y233NLQjV1/vXnz5vHWW28xf/581q5dy5tvvtmGtW4bnaPjvRUaHhYh3dRCiEB1hhZse8jMzGT58uWsXbuW0NBQMjIySE9Pb+hCbkprjVInT7Bteqy6urrZZ2FhYQ37Dz74IJMmTeLDDz9k//79DePrLZ13/vz5XHLJJdhsNq688spzcszZr5bDBAgLMhNsMUkyFkKIDlRaWkp0dDShoaHs3LmTb775hpqaGlatWsW+ffsAGrqpp06dyvPPP9/w3fpu6sTERLKzs/F4PA0t7Jau1bWrcYft66+/3nB86tSpLFy4EJfL1ex6ycnJJCcn84c//KFhHPpc43fJWCklC38IIUQHmz59Oi6XiyFDhvDggw8yduxY4uPjWbRoEVdccQXp6elcffXVAPz2t7+luLiYwYMHk56ezsqVKwF49NFHmTFjBpMnTyYpKanFa/3617/m/vvvZ/z48bjd7objN954Iz169GDIkCGkp6ezePHihs+uueYaunfvzsCBA9vpJ/DDnHtt9TYQFx5MvrSMhRCiwwQHB/P555+f8rOLL7642Xu73c4bb7zR7Fh5eTmzZs1i1qxZJ32/aesX4Pzzzycnp/HO2UceeQQAi8XC008/zdNPP33SOdasWcNNN93kVV18wS+Tcbw9iMMl1WcuKIQQwu+NGDGCsLAwnnrqKV+H0iK/TMZx9mC25Jb6OgwhhBDngI0bN/o6hDPyuzFjMB6jWFRRi8dz4qqdQgghxLnHL5NxnD0Yt0dTUuX0dShCCCHEGfltMga511gIIUTn4N/JWFbhEkII0Qn4ZTKODzceFiG3NwkhhOgM/DIZx4YZLWNZ+EMIIc5NTZ/QdKL9+/czePDgDozG9/wyGUeGWLGYlIwZCyGE6BT88j5jk0kRaw+SZCyECEiPffsYO4t2tuk5U2NSuXf0vS1+fu+999KzZ09uu+02AB5++GGUUqxevZri4mKcTid/+MMfmDlzZquuW11dza233sqGDRsaVtiaNGkS27dvZ/78+dTW1uLxePjggw9ITk7mqquuIjc3F7fbzYMPPtiwBOe5zi+TMRiTuOSZxkII0TFmz57Nr371q4Zk/O6777Js2TLuuusuIiIiKCgoYOzYsVx66aWnfLJSS1544QUAsrKy2LlzJ1OnTiUnJ4eFCxfyy1/+kmuuuYba2lrcbjdLly4lOTmZzz77DDAeKNFZ+G0yjrUHS8tYCBGQTteCbS/Dhg3j+PHjHDlyhPz8fKKjo0lKSuKuu+5i9erVmEwmDh8+zLFjx+jSpYvX512zZg133HEHAKmpqfTs2ZOcnBzOP/98/vjHP5Kbm8sVV1xB3759SUtL4+677+bee+9lxowZXHjhhe1V3Tbnl2PGAHH2IJnAJYQQHWjWrFm8//77vPPOO8yePZu3336b/Px8Nm7cyObNm0lMTDzpOcVnovWpV1KcO3cuH3/8MSEhIUybNo0VK1bQr18/Nm7cSFpaGvfffz+///3v26JaHcJvW8bxduPJTS09bFoIIUTbmj17NjfddBMFBQWsWrWKd999l4SEBKxWKytXruTAgQOtPueECRN4++23mTx5Mjk5ORw8eJD+/fuzd+9eevfuzZ133snevXvZunUrqampxMTE8LOf/Qy73X7S057OZX6bjOPswdS6PJTXuIiwWX0djhBC+L1BgwZRXl5O165dSUpK4pprruGSSy5h5MiRDB06lNTU1Faf87bbbuOWW24hLS0Ni8XC66+/TnBwMO+88w5vvfUWVquVLl268NBDD7F+/XruueceTCYTVquVv/3tb+1Qy/bhv8m4buGPgvIaScZCCNFBsrKyGvbj4uJYu3btKcs5HI4Wz5GSksK2bdsAsNlsp2zh3n///dx///3Njk2bNo1p06adRdS+57djxg0Lf1TIuLEQQohzm1+0jLPys3jq6FOkVqTSJcyYpSfrUwshxLktKyuLefPmAeDxeDCZTAQHB7Nu3TofR9bx/CIZRwZHsr92P1/u/5JrB10LNOmmltubhBDinJSWlsbmzZsBKC8vJzw83McR+Y5fdFP3iOhBt6BufHHgi4ZjMaFBKAX5cnuTEEKIc5xXyVgpNV0ptUsptVspdd8pPs9QSpUqpTbXvR5q+1BPb1joMLbmb+Wo4ygAFrOJmFBZElMIIcS574zJWCllBl4ALgYGAnOUUgNPUfQ/Wuuhda8Ov9N6WOgwAL488GXDsVh7EIWSjIUQQpzjvGkZjwZ2a633aq1rgSVA61b67gDx1ngGxAxoloxlfWohhBCdgTcTuLoCh5q8zwXGnKLc+UqpLcAR4G6t9fYTCyilbgZuBkhMTCQzM7PVAbfE4XDQx9OHT4o+4Z/L/0mMJQZPZTWHSj1tep1zjcPh8Ov6nUog1hkCs96BWGc4u3pHRkZSXl7ePgG1g6SkJI4ePdrw3u12d6r4vVFdXe31v6M3yfhUa0meuFjoJqCn1tqhlPox8BHQ96Qvab0IWAQwcuRInZGR4VWQ3sjMzOTW4bfyyYefUJ5UzhWDrmB1+Q6y1h+kLa9zrsnMzPTr+p1KINYZArPegVhnOLt6Z2dnd7rZyE3j9dVsapfLhcXSPjcW2Ww2hg0b5lVZbyLIBbo3ed8No/XbQGtd1mR/qVLq/5RScVrrAq+iaCPdI7o3dFVfN+g64sKDqKh1U1XrJiTI3JGhCCGEz+T96U/UZLft84yDB6TS5YEHWvy8LZ9n7HA4mDlz5im/9+abb/Lkk0+ilGLIkCH8/e9/59ixY9xyyy3s3bsXgL/97W8kJyczY8aMhpW8nnzySRwOBw8//DAZGRmMGzeOr7/+mksvvZR+/frxhz/8gdraWmJjY3n77bdJTEzE4XBwxx13sGHDBpRS/O53v6OkpIRt27bxzDPPAPDSSy+RnZ3N008//YN+vt4k4/VAX6VUL+AwMBuY27SAUqoLcExrrZVSozHGogt/UGRnaWrKVJ7d9CxHHUeJq1uFq8BRQ/eYUF+EI4QQAaEtn2dss9n48MMPT/rejh07+OMf/8jXX39NXFwcRUVFANx5551MnDiRDz/8ELfbjcPhoLi4+LTXKCkpYdWqVQAUFxfzzTffoJTi5Zdf5vHHH+epp57ikUceITIysmGJz+LiYoKCghgyZAiPP/44VquV1157jRdffPGH/vjOnIy11i6l1O3AF4AZeFVrvV0pdUvd5wuBWcCtSikXUAXM1i0996qdTes5jWc3PcuXB76ke/h0QJKxECKwnK4F217a8nnGWmseeOCBk763YsUKZs2aRVxcHAAxMTEArFixgjfffBMAs9lMZGTkGZPx1Vdf3bCfm5vL1VdfzdGjR6mtraVXr14ALF++nCVLljSUi46OBmDy5Ml8+umnDBgwAKfTSVpaWit/WifzqqNca70UWHrCsYVN9p8Hnv/B0bSBhq7q/V9yb7rRrSEzqoUQov3VP884Ly/vpOcZW61WUlJSvHqecUvfa80jcS0WCx6Pp+H9idcNCwtr2L/jjjtYsGABl156KZmZmTz88MMALV7vxhtv5E9/+hOpqanMnz/fq3jOxC9W4DrR1JSpbC3YirIafxntPFp2hm8IIYT4oWbPns2SJUt4//33mTVrFqWlpWf1POOWvjdlyhTeffddCguNUdD6buopU6Y0PC7R7XZTVlZGYmIix48fp7CwkJqaGj799NPTXq9r164AvPHGGw3Hp06dyvPPN7Yz61vbY8aM4dChQyxevJg5c+Z4++M5Lb9MxtN6Go/Q2lSwmgv7xvHmNweocbl9HJUQQvi3Uz3PeMOGDYwcOZK3337b6+cZt/S9QYMG8Zvf/IaJEyeSnp7OggULAHj22WdZuXIlaWlpjBgxgu3bt2O1WnnooYcYM2YMM2bMOO21H374Ya688kouvPDChi5wgN/+9rcUFxczePBg0tPTWblyZcNnV111FePHj2/ouv6h/OJBESdq2lV9y4QfM++Vb/nXd0e4alT3M39ZCCHEWWuL5xmf7nvXXXcd1113XbNjiYmJ/Otf/zqp7J133smdd9550vET7/2dOXPmKWd52+32Zi3lptasWcNdd93VUhVazS9bxtDYVd27Sy0DkyJY9J+9eDw+mVMmhBDCT5SUlNCvXz9CQkKYMmVKm53Xb5NxfVf18oPLuXlCb3Yfd7Bi53EfRyWEEKJeVlYWQ4cOZejQoYwfP56hQ4cyZsypFng8d0RFRZGTk8N7773Xpuf1y25qaN5V/fr0eTy+bCeLVu/lRwMTfR2aEEK0i9bMNj4X+PPzjFt7d6/ftowBpqVMY2vBVvKr8vj5Bb34dn8R3x08/b1nQgjRGdlsNgoLC1udBETb01pTWFiIzWbz+jt+2zIGIxn/9bu/8quVv+KP45/k2a8sLFq9l7/9bISvQxNCiDbVrVs3cnNzyc/P93UoZ6W6urpVyetcZ7PZ6Natm9fl/ToZdwvvxrOTnuX+/9zPz7+8hinD7uBf37jYX1BBSlzYmU8ghBCdhNVqbVg5qjPKzMz0+qEK/sivu6kBJnafyD9m/INYWywrS/6ILXY1L/1nj6/DEkIIIRr4fTIG6BnRk8U/WcyUnlOwxC/lw8OPcahExo6FEEKcGwIiGQOEWkN5auJTXNv/Nkz2LOZ+dg255bm+DksIIYQInGQMoJTinrG3MkAtoKSmgAWZ/4vL4/J1WEIIIQJcQCXjevdMuJSqo5eRXbSDt7Pf9nU4QgghAlxAJuNRKdFM7PYjXOUDeXbTXzlYdtDXIQkhhAhgAZmMlVL85ephJLnm4nQp7lv1IB7tOfMXhRBCiHYQkMkYINxm5fV5F2EqvoSsok28tf1dX4ckhBAiQAVsMgboERvKizNvw115Hk9teJrDZUd9HZIQQogAFNDJGGDseXHcMeR+3NrF9Z/cJ+u6CiGE6HABn4wBbh0/hmH22eS5NvGbf//d1+EIIYQIMJKM67xy+QLCdC8+PvR/fL5jt6/DEUIIEUAkGdcJslhYOP0xlLmau1fdzcdZu3wdkhBCiAAhybiJoV0GcN+oBzGFHOD+dT/n2TVf+jokIYQQAcCrZKyUmq6U2qWU2q2Uuu805UYppdxKqVltF2LHumbQT3l92psEW4J5afc9/OLTp2VSlxBCiHZ1xmSslDIDLwAXAwOBOUqpgS2Uewz4oq2D7GgjktL48sp/EquGsbrwNWa8ewOl1aW+DksIIYSfsnhRZjSwW2u9F0AptQSYCew4odwdwAfAqDaN0EfiwqJYfs0rXPPek+yoWszU9y7noXF3ExYUCtCstdwlrAsDYgf4KlQhhBCdnDfJuCtwqMn7XGBM0wJKqa7A5cBk/CQZA1gtZt6Z/Wv+9+NUvsh/gvvW3Nti2esHXc+dw+/EarKe9fUKqgqItcWilDrrcwghhOh81JnGQ5VSVwLTtNY31r2fB4zWWt/RpMx7wFNa62+UUq8Dn2qt3z/FuW4GbgZITEwcsWTJkjariMPhwG63t9n5TvTpPgf/OngYmxl+3DuIQbFmFAqNZq1jLWsca+gV3Iv5cfOJtkS3+vyrylbxQfEHjLWPZU7MHK8TcnvX+1wUiHWGwKx3INYZArPegVLnSZMmbdRajzzpA631aV/A+cAXTd7fD9x/Qpl9wP66lwM4Dlx2uvOOGDFCt6WVK1e26flOJftoqb7kr//RPe/9VN/y9w36WFlVw2ef7/1cj3l7jL7gHxfo1YdWe31Ot8etH//2cT349cF6+vvT9eDXB+vF2Yu9/n5H1PtcE4h11jow6x2IddY6MOsdKHUGNuhT5ERvZlOvB/oqpXoppYKA2cDHJyT0XlrrFK11CvA+cJvW+qPW/81wbkvtEsE/bx3HvdNT+WrncaY+s5oPv8tFa830XtN5Z8Y7JIYmcttXt/GXjX/B5XGd9nzVLuOe5jd3vMnc1Ll8fPnHZHTL4LFvH2N93voOqpUQQghfO2My1lq7gNsxZklnA+9qrbcrpW5RSt3S3gGeayxmE7dmnMfSOy+kd1wYd72zhZ+/vp59BRX0jOjJWz9+i1n9ZvHKtle4btl1fLT7IwqrCk86T1F1ETd+eSPLDyznnpH3cN/o+7CarPz5wj/TM6InCzIXkFue64MaCiGE6GjeTOBCa70UWHrCsYUtlL3+h4d17uuTYOe9W8bxxn/389SXu5j6zCquH5fCHVP68rvzf8fIxJE8vfFpHvz6QRSKtLg0JnSbwMTuE7GZbdz21W0crzzOUxlPcVHPixrOaw+y89zk55jz2Rx+ufKX/P3ivxNqDfVhTYUQQrQ3WYHrBzCbFD+/oBcr78ng8mFdeXnNPiY9kcnidQeZnvJjls9aznuXvMcvhv4CgBc2v8CVn1zJpR9diqPWwSvTXmmWiOv1jOjJkxOeZHfJbn779W/xaE9HV00IIUQH8qplLE4vIdzG47PSufb8FP7fJ9t54MMs3ly7n4dmDGRcn1RSY1L5n/T/oaCqgDWH17CraBdzU+fSPaJ7i+cc13UcC0Ys4MkNT7Jo6yJuSQ+4EQEhhAgYkozb0OCukbz7P+ezNCuPPy3NZu7L6xidEsOtk84jo188cSFxXNbnMq/Pd+3Aa8kpzuGFzS8QYgnh6v5XY7PY2rEGQgghfEG6qduYUoqfDEniq/+dyEMzBnKouJL5r63nJ8+t4ZMtR3B7vF/nWinFQ+c/xLjkcTy54UmmfTCNhVsWUlojS3MKIYQ/kWTcTmxWMz+/oBer7pnE47OGUO1yc8c/vmPyU8aYcrXT7dV5gs3BLPzRQl6b9hqD4wbzwuYXuOj9i3js28c46jjazrUQQgjREaSbup0FWUxcNbI7Px3ejS+35/F/mXt44MMsnv73LuaNTeFnY3sQaw8+7TmUUozsMpKRXUaSU5zDG9vfYMnOJSzZuYRkazKvff4aQeYg42UKwmq2khKRwpzUOUTbWr8amBBCiI4lLeMOYjYpLk5L4uPbx7P4xjGkdY3kmeU5jHt0BQ98mMWefIdX5+kX3Y8/XvBHll6xlGsGXEOYKQyzyUyVq4r8ynz2le5jW8E2Fm5ZyPQPpvPcpuekW1sIIc5x0jLuYEopxvWJY1yfOL4/Vs4ra/bx/sZcFq87yI8GJHDduBTGnxeHyXT6tamT7EncPepuMisyycjIOOnz3cW7Wbh1IS9lvcTinYv52YCfMW/gPCKDIwEory1na/5Wvjv+HZvzN5Nfmc+CEQuY2H1ie1RbCCHEaUgy9qG+ieE8+tMh3D2tP39fe4C3vjnA8uxv6RkbypzRPbhyRLczdmG3pE90H56c+CQ3D7mZhVsW8uLWF1mcvZgJ3SeQU5zD7uLdaDQmZaJ/dH8Abl9xOzel3cQvhv4Cs8ncllUVQghxGpKMzwFx9mDuuqgft006j2Xb8nh73UEe/XwnT3+Zw/TBXZg7pgdjesWc1aMV+0X34+mMp9lVtIu/bfkba4+sZUDMAKb2nMrQhKGkxaURZg2j2lXNn9b9iZeyXmJbwTYem/CY1+PNWmuKa4rZV7qPg2UHiQuJY3TSaILNZ/eHhBBCBBpJxueQYIuZmUO7MnNoV74/Vs7b6w7ywaZcPt5yhO4xIfwkLZkZQ5IYlBzR6sTcP6Y/f5n0lxY/t1ls/H7870mPT+dP6/7E1Z9ezdMZTzM4bnCzcpXOSrYXbierIIs9JXvYX7af/aX7Kasta1YuxBLCmKQxTOw2kQu7XkhiWGKr4hVCiEAiyfgc1TcxnIcvHcS901P5LOsoH285wkv/2cvCVXtIiQ3lJ0OS+Elacv0jLNvMT/v9lNTYVBasXMC1n1/L/478XyKCItiSv4Wt+VvJKc7BrY3bsuJD4ukV2YvpKdNJiUwhJSKFnhE9OVh+kFWHVrE6dzWZhzIBGBAzgLFJYxkQO4DUmFR6RvTEpGT+oBBCgCTjc15IkJlZI7oxa0Q3iipq+XJ7Hp9lHWXhqr28sHIPXUIVVzp3MWNIMv0S7WfVlX2iQbGDeGfGO9y35j4e/fZRAMKsYaTFpXFD2g2kx6eTFpfWYjd2j4geXND1Ah7QD7C7ZDerc1ezOnc1b2W/hdPjBCDUEkr/mP4MiBnAyC4jmdx9soxTCyECliTjTiQmLIjZo3swe3QPCh01LNuex1urdvDCyt38dcVu+iTYmTEkiRlDkuiTEP6DrhVli+KFyS+wLm8d8SHx9I7s3epkqZSib3Rf+kb35Ya0G3C6newt3cuOwh3sLNpJdlE2H+7+kMU7F5MSkcLNQ27m4l4XYzF1zH+WueW5mJWZJHtSh1xPCCFaIsm4k4q1B3PNmJ50rdrHoBHns2x7Hp9uOcKzX33PX5Z/T//EcKYNSmTa4C4MTGr9GDOA2WRmXPK4NovZarbSP6Y//WP6Nxxze9ysOLSChVsW8sCaB/jblr9xU9pNzDhvBlaTtc2uDcZEs5ziHJYfXM7yA8vZXbIbgMTQRIYnDGdY4jCGJQyjb1RfaaULITqUJGM/EB8ezLyxPZk3tifHy6pZmnWUz7fl8fzK3Ty3YjfdokOYPqgL0wd3YXiP6DPew9yRzCYzF/W8iCk9ppB5KJOFWxby0H8f4sWtL3JV/6uwW+0nfWePYw89SnrQK7LXGf/IcLqdbC3YysqDK/nq4FfkOnIxKRPDEobx61G/xqzMfHf8OzYe38jn+z8HwG61MzRhKGOTxjI2aSx9o/vK+LYQol1JMvYzCRE2rh/fi+vH96LQUcPy7GMs25bHm2sP8PKafcTZg5jQN56J/eO5sG88MWFBvg4ZAJMyMbnHZCZ1n8Tq3NUs3LKQZzY+02L5xf9aTEJoAmOTxnJ+8vmMTRpLXEgc1a5qsgqy2JC3gQ3HNrAlfws17hosJgtjk8ZyY9qNZHTPIDYktuFccwfMRWvN0YqjbDq+iU3HNrE+bz1PHn4SgBhbDGOSxnB+0vmMSBxBV3vXDm85f7znY7aUb2GUcxRh1rAOvbYQov1JMvZjsfZgrh7Vg6tH9aC82snKXfks33GMlbuO88/vDqMUDOkWxcR+8UzsF0da1yiCLL5tASqlmNh9IhO6TaCwurDZbPH6VvCKNSsw9TSx9shaVuWu4uM9HwPQI7wHRyuO4vQ4UShSY1K5st+VjEgcwZikMYQHtTyOrpQi2Z5Msj2ZGb1nAJBXkcc3R78xXke+4fN9RsvZarLSI7xHsxnkqTGpDIgd0OY/D4/28NSGp3hzx5sALH1vKTP7zGRO6hx6RvRs8+uda2o8Nb4OQYgOIck4QITbrFyansyl6cm4PZqsw6Ws2pXPqpzjPL/ie5776nuCLSaGdItkRM8YRvSMZniPqLNeAeyHUkoRFxJ3ys8SrAlk9MtgVr9ZeLSHnUU7WXtkLVvztzK5x2RGJo5kWOIwIoIiflAMXcK6cFmfy7isz2VordldspusgqyGe6v3lu5lVe4qXB4XAGOSxvDLYb8kLT7tB123Xq27lt+s+Q3L9i9jTuockkqS2BW6i3d2vcPb2W8zvut4rkm9hvFdx/tdN7rT7eSx9Y/x7qF3yfkuh9vSb5NxfOHXJBkHILNJMbR7FEO7R/HLH/WlpLKWtXsK2XigmI0Hi3llzV4WrjJapL3iwrhoYCIzhiSR1jWyTW6daksmZWJg7EAGxg5s1+s0nRnelMvj4rDjMKsOreLlrJeZu3Quk7tP5vZht59UtqlKZyUhlpAWf56lCll8KAAAHwJJREFUNaX8auWv2HBsAwtGLOD6QdezatUq5l84n7tH3s17Oe/9//buPDjO+kzw+PeR+lJ3S637si5Llm8bA7aBGIw5Es4EdoAcTJIZJgwLk5mahGzl2EqyJCkyQM3uJDWVcIQ4y2YSIAQSDCExhERcMY4PfAO2JFvWaZ0tqS+pJf32j7fdSJZsZJDcVvfzqXrr7fftV6/ex5L89O/mqXef4p9e/ieqsqq4fcXtXFt97Yx3ekuE7nA3d9fdzVudb1HlqOKRPY+wq3MX96+//6Qf0JSa6zQZK7LdDq5ZUcI1K6whPpHoKPta+9nR1MebjT387I3DPPJqIxW51mQj168s+cA9tJONLc1GZVYln1/2eW5aeBM/P/BzHtv/GDdtuonrq6/n9pW3ExmJcKjvkLX5rX1XuIuKzAquqLiCyysuZ2XBynjptiPYwV1/vIsjA0e475L7uK76ugnfMz8jn7vOuYvbl9/OS00vsXHfRr75xjd5cPeDfGHFF7ih5gYc6WdHX4DTtatzF3fX3U0gGuCB9Q+Q0ZSBv8zPvW/ey82bbub+9fdzQckFp33fgeEBXOmuOfvvopKfJmM1icuezuqqXFZX5fLfL62hPxRl84EOfrennZ+82siDdQ1U53u4pDaf86usKu152RmJfuyE89g93HnOnXx60afZuH8jv3z7lzzX+Fz8fUeag5rsGi4qvYgybxm7unbx8wM/52f7f0ZBRgGXV1zOqsJV/Mf2/yA0EuLhKx9mbcnak34/e7qda6uv5Zr51/Bqy6s8vOdhvrvluzy8+2FuW34b182/jtZgKw3+Bur99TT4G2jwN9AT7uHCkgu5svJKNpRviK/klWhPHXyK72/9PsXuYh688kEW5S6irqmOGxfcyLK8ZXzlla/wjy/+I3etuos7VtwxrWrrMTPGk+8+yQ92/IAiTxE/uOwHVPuqz0A0J9caaOXJd57kucbnqMqq4gsrvsC60nX64TbFaTJW78vntvPJ1eV8cnU5fcFh/rC/gxf2tvPUjhYe29IEQInPxfmVOayuzOH8ylwWl2RiT0+udszpynZlc/f5d/PZJZ/lxSMvUugupDanlvLM8kkTmgwMD/Bqy6v86eif2NSwiSfffZJCdyGPXfMYC3MWTuv7je/0tqV9Cw/vfpj7/npffPY0sDqdVfmqWJm/Eq/Dy2utr1HXUodNbKwpXsOVlVdyecXlp1UNHB2L0j/Ujz/ixz/kp3+4n/4haxs1o3jsHtw2N267G4/Ng9vuxpHuwBiDwWCMYYwxjDH8tv63PH3oadaVruP+9fdP+oBQm1PLE9c9wffe/B4/3vVjdh7byZfP/zJLcpecNIk1+Bu45y/3sKtrF2uL11Lvr+fW393Kvevu5YrKK6Yd5/h4G/2NBKIBzis877SSpzGGLe1bePydx3ml+RXSJI2L513M271vc9cf72JJ7hL+YcU/8NGKj2rb+Bl2oOcAbYE21s1bR4YtcYUKmem5jadr9erVZvv27TN2v7q6qdf1TXaJjHtkdIx3OgbZfqSX7U197Gzqo60/AoDLnsY5ZdmxjmA5nFeZM2PDqJL1Zx0ZibCraxcLcxaS68qd9P7pxL2tYxu7u3ZTmVVJTXYNFZkVEz4IGGPY37Ofl5pe4o9Nf+To4FEA8lx5lGWWUZZZRnlmOWXeMko8JfQP99M82BzfWgZbaA+2M2bGZiR2YMrlO0+M2RjDM4ee4b6/3kdkNMKC7AV8vObjXDf/uvhiJMOjw/x07095ZO8jeOwevrbma1xffT3HQse4u+5u9nbvfd+lQkfGRjjUd4gDPQfi28G+gwyPDQOwrnQd377o25R6S08ZU2Qkwm/qf8Pj7zzO4f7D5DhzuHnhzXxy0Scp9hQTHY3yfOPzbNy3kSMDR6jMquS2Zbcx2jTK4lWLGRgeYGBowNoPD+BMd1KbXcvC3IXkufJOuzQdHY3SNNBER6iDcwvPPauGyZ3O73coGqJpoIlsZzbFnuLT/ncwxrD92HZ+sucnbGnfAlg1W1dVXcUnaj5x2h+2ToeI7DDGrJ50fjrJWESuBn4IpAOPGmPuO+H9G4DvAWPACPAlY8zrp7qnJuOZcbbF3eoPs7Opj51HreS8v22AkTHrd6w632OVnqus0nNNgecD/cKfbTGfKbMVtzGGQ/5DvNbyGkcHj9Iy2ELLYAsdoY5JyTbbmW0l6ViyLswoxOf04XP6yHZmx/dpkkZoJEQwGiQUDREeCROMBhkeHUZEEGTCviCjYMLMbO8Xc/9QP5uPbOa5hufY1bULQbiw5EI2lG/gV+/+iob+Bq6dfy1fXfPVCWPKh0eH+f7W7/P0oaf5SOlHuP+S+8l2ZQPQG+nljdY3eLXlVd5oe4PB4UHAmgRmSd4SluZaHQV7Ij3851v/iSB86fwv8alFn5rUmz0yEuHXB3/NT/f9lO5wN8vzlnPrklv5WNXHplxa9PhMdI/ufZQDPQem9XPLceZQm1PLwpyFlGWW4Uh3YBMbtrT3tuholMMDh+NNFEcHjjJirN7/bpub66qv4+aFN896B8jpmOpnbYzh8MBhDvYd5FDfIer76jnkP0TLYAsG6/8Vt81NTXYN1b5qarJrqMmuoSyzjGJ3MW67e8L9xsyY1dly36Ps6dpDniuPzy/7PEvzlvK7xt+x+chmwiNhyjPL+XjNx/lEzSeY5503o3F+4GQsIunAQeCjQAuwDfiMMebAuGu8QNAYY0RkJfArY8ziU91Xk/HMONvjDg+PsjfWGczaeukLWYtF5HocnFeRw9r5OaxbkM+S4qxpzQ52tsc8W8503NHRKG3BNtoCbficPsozy085Vns2TCfmpoEmnm98nucanqM10Eqxp5hvXfgt1petP+nXPH3wae7dei+F7kKur76eLe1b2Nu1F4Mhz5XHJWWXcFHJRSzPX05ZZtmkZNsaaOU7f/kOW9q3cF7hedzzkXuY75s/KQmvLV7LnefcyZriNdOK93iJrW57HResuoAsR5a1Oa19MBqMdwSMJyh/PeGR8EnvmSZplGeWU+OzElV1djU5zhx+f/j3bD6ymchohGV5y7h54c1cM/+aaZWWg9Egdc11vNT0Ev4hP/kZ+ZO2HGcOGbYM3Ha3tbe5saefvLd/XV0dl156KUcGjrCtYxtb27ey/dh2eiO98TgqsypZkL2A2pxa5vvmMzA0QL2/nkZ/Iw39DXSHuyfc0+f0UewupsRTQpGniB3HdlDvr2eedx63LbuNG2tvnPDhKBQN8fLRl3m2/lm2dmxFEDbftHlG56//MMn4IuAeY8xVseNvABhj/u0U1280xpxyBgRNxjNjrsVtjKGhK8iOpl62HbES9OHuIAD5Xgcfqcnnktp8LqktoNjnmvIecy3mmZKKcZ9OzMdL+GXeskkloqns7drLl+u+TGeok+X5y7mk7BLWz1vPkrwl0xq3bYzh2YZneWDbAwyNDHHDghv4c/OfP1ASPtHpxD1mxuiL9DEyNsKIGWF0bJSRsRGiY1HSJZ3yrPIpS+Ng9Vl4vuF5njr4FPX+etw2NysLVrIox5pDfmHOQqp91djT7QSjQV5pfoXNRzbzeuvrDI8NU5hRSHlWOT3hHrrD3QSigVM+q01sZNgzyLRn4nF48Nq98a2js4MW00JXuAuAQncha4vXsqZ4DUvzljLfN/+kcRzXP9RPY38jLYMtHAsdoyPY8d4W6qDYXczfL/97rq66+n0XpGkLtLGlbQs3LbzplNedrpMl4+l04JoHNI87bgEmjS0Qkf8G/BtQCFx34vtKgdXZaEGhlwWFXj61pgKAjv4Ir9d38/qhLl6v72HT7jbAGuO8uDiT2qJMFhZ5WViUyfz8s6eNS51dRGTand4AVhSs4IW/eYFQNBSvqj7d73fjghu5eN7F3PvmvTx18CnWFq/lgfUPfOAk/EGkSdqEqvjTkeXI4tYlt/KZxZ9hT/ceNtVvYl/PPh5/5/F4+7gtzUZVVhXNg80MjQ5RmFHILYtu4aqqqzin4JwJH1zCI2G6w930hHvwD/kJj4QJj4TjTRWhkRChaIhANEBgOEAwGqQn0sPRwaMMDg1yQfkFrClZw9ritVRkVpx2M5bP6ePcQmvBlw+r1Fs644n4VKZTMr4FuMoYc3vs+HPAWmPMv5zk+vXAt40xV07x3h3AHQBFRUXnP/HEEx/y8d8TCATweicvKpDski1uYwwtAcO+7lEO9Y3SGhijM3S8dQjSBQozDDU5dqp9aVT70ijLTMN2Fi1+MVuS7Wc9HXMp5vBYmIy0memNm+i4R80ondFO2qJttA630h5tJ8+Wx7nuc5nvnD8rM74lOuYz5bLLLvvAJeMWoHzccRnQdrKLjTGvikiNiOQbY7pPeO8R4BGwqqlnssotFavwIDXijkRHaegKcOhYgIPHBnlj/xHe9qfxeqv1yd1hS2NZaRbnlGWztDSLJcVZ1BZ5cdmTa4hIKvysT5SKMUNqxp2KMY83nWS8DagVkflAK/Bp4NbxF4jIAqAh1oHrPMAB9Mz0w6rU5LKns6zUx7JSa+xpnauDSy+9lJa+MLtb/Oxu9rO7uZ8ntzUTjo4CkCbHq7mzWFycaQ2xqsxJugStlEoO75uMjTEjIvLPwGasoU0bjTH7ReTO2PsPATcBnxeRKBAGPmUSNYBZpQQRoTzXTXmum+tXWmM9R8cMTT1B3u0Y5O2OQd5pH2BfWz+/29sOWCXo8ytyWLcgj4tq8llZ5kvZiUmUUmeXac3AZYx5AXjhhHMPjXt9P3D/zD6aUqcnPU2oLvBSXeCNz7MNMBiJsv1IH2/Ud/OXhh7+/cWDwEG8ThtLS7KoyHNTmeumMt9j7fPcZLt1DmOl1Jmj02GqpJfpsnPZ4kIuW1wIQG9wmDcbe/hLQzcHOwK8dqiLXw9MXDc3z+NgaWkWS0uy4vvqAi/pKdBRTCl15mkyVikn1+Pg2hUlXDuu9BweHuVob4imniBNPSHqOwPsb+/nZ28cYXjUmoXKZU9jUXEWy0qzWF7qY/m8LBYWZWo7tFLqQ9NkrBSQ4UhnUXEmi4onzjAVHR2joSvAgbYBDrRZbdDP7W7jl1utuZxtada46WWlPhYUeqkp8LCg0EtFrhubtkcrpaZJk7FSp2BPT4v1yM7ib86zzhljaOkLs6+1n31t/exvG+C1Q108vbNl3NcJlXkeFhR4WVJilaaXzcuiOMulS+UppSbRZKzUaRrfk3t8R7GBSJTGriD1nQEaugI0dAZ499ggmw90cHxsQa7HwdJYcq4tyqSmwEN1gRdfxsnn7FVKJT9NxkrNkCyXnVXl2awqnzi1YnBohLfbB9jfNsD+WEl64xuHiY6+N/ov3+uMJ+aynAyKs1wUZbko9jkpzHKR6dQ/VaWSmf6FKzXLPE4bq6tyWV313hrF0dExmntDNHQFaegK0NgVoKEryB/2tcdXtRrP7Ugn32m4qHsPy8t8rJjnY3Gxdh5TKlloMlYqAezpafEx0R+laMJ7oeERjg0McWwgEt86+ofY+k4Tmw908OR2a90WW5pQW5TJkuJMynLdlOVkUJaTQXmOm2KfSyc0UWoO0WSs1FnG7bAxP982aYWquszO+DSg+1r72Rvbth7u5be7WhkbN+ddmkB5rptV5dmcW57NuRU5LCnJwmHTBK3U2UiTsVJzyMk6j0VHx+joj9DcF6KlL0xLX5hDxwbZ2tjLs7usdV2ctjRWzPOxsiyb8twMSrMzKPVlUJrtItfj0F7eSiWQJmOlkoA9PS2epE/U3h/mraN+djb18Vazn19sbWJoZGzCNU5bGqXZGbGqbqvKuzzXTXnsON+ryVqp2aTJWKkkV+LLoGRFRnzGMWMMPcFh2v0RWv1h2vvDtPnDtPkjtPSFeLGtg57g8IR7ZDptzC/wMD/f2qoLvFTne6jMc5Pp0mFZSn1YmoyVSjEiQr7XSb7XyYoy35TXBIdGYtXdIY72hjjSHaSxO8iOpj427W5j/Jps+V4HFbluqvI8VOZZCfp4NXhhpkvn81ZqGjQZK6Um8ThtU04PChCJjtLUE6KxK8CRnvfm836zsYdn3mqdcG16mlCU6aQkO9ZGne2ifFw1+LzsDB2epRSajJVSp8lln3oeb7ASdXNviBa/VfXd7o/Q1m/t97T42bwvEl9447jCTKc10YkvNtFJliv+ujM0xvDImPYCV0lPk7FSasa47OnUFmVSWzQ5UQOMjRk6B4divb5DNPeGae4N0eoP807HIK+820VweHTC13z9td9T4sugPNcaQ12e66Yi16oKL8txU+B1kqZV4WqO02SslDpj0tKEYp9V8l0zbkay8QYjUTr6I3QMRPjz1l14Cys42huiuS/MKwe76BycuPa0w5YWn+zkvV7g7njyznbbtSe4OutpMlZKnVUyXXYyXXZqizIZbbWzYcOiCe9HoqPxUnVLn5Wkm3ut8dW7W/z4T5hO1Ou0TUjSx18f33t13m91FtDfQqXUnOKyp7OgMJMFhVNXhQ9EorT0hmnuC8WTdHOv1dHs9UPdhKMTq8Gz3XbmZWdYW461P56oawq82sFMnRGajJVSSSXLZWdpqZ2lpVmT3jPG0BscthJ0rHTd6rcS9uHuIK/XdxMa12YtAmU5GSwo8FJT4GVBoZf5+R5yPQ58bju+DDtOmyZr9eFpMlZKpQwRIc/rJM/r5JwTlroEK1n7Q1Fa+sI09VprUx/f3mjoYfiEmcsAMuzpZLvt5LgdzM/3sKDQStq1RVbi1mStpkOTsVJKxYgIOR4HOR7HpAlRRsdMfBIUfyiKPxylPzQcf90TGGJfWz8v7GuPT4qSJlCR645PhlIR6wlemeehItdNhkMTtbJoMlZKqWlIT5NYUvWc8rpIdJTGriD1XQHqjw1S3xWgqSfEzqY+BodGJlyb7bZTmOmkINNJYaYr/rq7fQRnQw+FWU4KM514nTbtEZ7kNBkrpdQMctnTWVqaNanN+ngV+NHeEE29Vueyjv4InYMROgeH2Hakl87BoXhV+EO734x/bYY9ncIsJ0WZLmuf5aIovn9vohTtbDZ3TSsZi8jVwA+BdOBRY8x9J7z/t8DXYocB4C5jzO6ZfFCllJrLxleBT9VeDVbCHgiP8PyfXqNq8UorUQ8M0Tk4RNfgEMcGIuxvG+Dltzsn9QoHyPU4KPG5Yps1q1lptvW6JDa+W9uwz07vm4xFJB34EfBRoAXYJiKbjDEHxl12GLjUGNMnItcAjwAXzMYDK6VUshIRfG4787xprFuQf9LrjDEEhkY4NhDh2MAQHf0Ra/Wt/gjtfms9621H+ugPRyd9bZ7HQbHPFV8sJN/rIM/rIM/jJM/riFeZ53kcOrPZGTSdkvFaoN4Y0wggIk8ANwDxZGyM+cu4698EymbyIZVSSr1HROKTo5xsvDVYq291DETo6I/Q5g9b+/4IHf1heoLD1HcG6AoMTdlLPD1NyPc64m3Z1rhrDzUFXqoLvBRlObUdewZNJxnPA5rHHbdw6lLvF4Dff5iHUkop9eF5nDZqYmOkT+Z4KbsnMEx3wKoO7xwcileRdwWGaOuP8GZjz4R5wz2OdGoKvRRnufA6bXhdNrxOGx6njUyXjWy3g1KfK7aUphNbui72cSpixi9MOtUFIrcAVxljbo8dfw5Ya4z5lymuvQz4MXCxMaZnivfvAO4AKCoqOv+JJ5748BHEBAIBvN6T/8Ilq1SMOxVjhtSMOxVjhrMzbmMM/iFDe9DQHhyjPTBGe3CMgWEIjxjCI4bICIxOkVLSBLKdQp5LyMsQ8lxp5GUI+RlCXkYa+S4hGgmedTHPhssuu2yHMWb1ieenUzJuAcrHHZcBbSdeJCIrgUeBa6ZKxADGmEew2pNZvXq12bBhwzS+/fTU1dUxk/ebK1Ix7lSMGVIz7lSMGeZu3MYYhkbGCAyN0Bscps0fps1vVZG39Yfjxzs6w0RPyNpeu1CRn251Pjuh01lhrBd5ZhIP8ZpOMt4G1IrIfKAV+DRw6/gLRKQCeAb4nDHm4Iw/pVJKqbOeiOCyp+Oyp5PvdbLwJEtpjo4ZugaHaOmzls9s6Quz7UADaR4Xbf0Rdhztm7TgB4DLnhZvwy7McpLncZLrcUzaSn0Z+Nz22Q53Rr1vMjbGjIjIPwObsYY2bTTG7BeRO2PvPwR8G8gDfhz71DIyVTFcKaWUSh+3lObxRFEnLWzYsCZ+TXh4lPb+cGws9tCkYV7vdgzSE+yZMmmDNaFKZZ6Hqjx3fF/iyyDX4yDHbSfb7cBhO3vasac1ztgY8wLwwgnnHhr3+nbg9pl9NKWUUqkqw5FOdazn9qmMjI7hD0fpDQ7TGxymJ2BVjx/pCdLUE2JHUx+bdrcxVfcor9NGttseL1EfH95lvbaGfK1bkH9GxmbrDFxKKaXmLFt6WnzM9MkMjYzS3BumcyBCXyhKb2gYf3DY2oei9AStnuQHOwbpCQ4zNG6o1557PqbJWCmllPqwnLb0+Gpa78cYQ2h4lN5Ygs50npk0qclYKaWUihERPLHx0uW57jP2fc+e1mullFIqRWkyVkoppRJMk7FSSimVYJqMlVJKqQTTZKyUUkolmCZjpZRSKsE0GSullFIJpslYKaWUSjBNxkoppVSCaTJWSimlEkzMVEtZnIlvLNIFNM3gLfOB7hm831yRinGnYsyQmnGnYsyQmnGnSsyVxpiCE08mLBnPNBHZnoprKKdi3KkYM6Rm3KkYM6Rm3KkY83haTa2UUkolmCZjpZRSKsGSKRk/kugHSJBUjDsVY4bUjDsVY4bUjDsVY45LmjZjpZRSaq5KppKxUkopNSclRTIWkatF5F0RqReRryf6eWaLiGwUkU4R2TfuXK6IvCQih2L7nEQ+40wTkXIR+bOIvC0i+0XkX2PnkzZuEXGJyF9FZHcs5u/EzidtzOOJSLqIvCUiz8eOkzpuETkiIntFZJeIbI+dS+qYAUQkW0R+LSLvxP6+L0qFuE9mzidjEUkHfgRcAywFPiMiSxP7VLPm/wJXn3Du68DLxpha4OXYcTIZAb5ijFkCXAh8MfbzTea4h4DLjTHnAKuAq0XkQpI75vH+FXh73HEqxH2ZMWbVuKE9qRDzD4E/GGMWA+dg/cxTIe4pzflkDKwF6o0xjcaYYeAJ4IYEP9OsMMa8CvSecPoG4LHY68eAG8/oQ80yY0y7MWZn7PUg1h/sPJI4bmMJxA7tsc2QxDEfJyJlwHXAo+NOJ33cU0jqmEUkC1gP/BTAGDNsjPGT5HGfSjIk43lA87jjlti5VFFkjGkHK3EBhQl+nlkjIlXAucBWkjzuWFXtLqATeMkYk/Qxx/wA+CowNu5cssdtgBdFZIeI3BE7l+wxVwNdwM9iTRKPioiH5I/7pJIhGcsU57SLeJIRES/wNPAlY8xAop9nthljRo0xq4AyYK2ILE/0M802Ebke6DTG7Ej0s5xh64wx52E1tX1RRNYn+oHOABtwHvCgMeZcIEgKVUlPJRmScQtQPu64DGhL0LMkwjERKQGI7TsT/DwzTkTsWIn4F8aYZ2Knkz5ugFjVXR1WX4Fkj3kd8AkROYLV3HS5iPwXSR63MaYttu8EfoPV9JbUMWP9v90Sq/EB+DVWck72uE8qGZLxNqBWROaLiAP4NLApwc90Jm0C/i72+u+AZxP4LDNORASrXeltY8z/GfdW0sYtIgUikh17nQFcCbxDEscMYIz5hjGmzBhThfV3/CdjzGdJ4rhFxCMimcdfAx8D9pHEMQMYYzqAZhFZFDt1BXCAJI/7VJJi0g8RuRarrSkd2GiMuTfBjzQrRORxYAPW6ibHgP8F/Bb4FVABHAVuMcac2MlrzhKRi4HXgL281474P7HajZMybhFZidV5JR3rA/OvjDHfFZE8kjTmE4nIBuB/GGOuT+a4RaQaqzQMVtXtL40x9yZzzMeJyCqsjnoOoBG4jdjvO0kc98kkRTJWSiml5rJkqKZWSiml5jRNxkoppVSCaTJWSimlEkyTsVJKKZVgmoyVUkqpBNNkrJSaREQ2HF81SSk1+zQZK6WUUgmmyVipOUxEPhtb+3iXiDwcW2AiICL/W0R2isjLIlIQu3aViLwpIntE5DfH14oVkQUi8sfY+sk7RaQmdnvvuPVmfxGbDU0pNQs0GSs1R4nIEuBTWAsNrAJGgb8FPMDO2OIDr2DN1Abw/4CvGWNWYs1odvz8L4AfxdZP/gjQHjt/LvAlrHXCq7HmjlZKzQJboh9AKfWBXQGcD2yLFVozsCbWHwOejF3zX8AzIuIDso0xr8TOPwY8FZsXeZ4x5jcAxpgIQOx+fzXGtMSOdwFVwOuzH5ZSqUeTsVJzlwCPGWO+MeGkyLdOuO5Uc96equp5aNzrUfT/C6VmjVZTKzV3vQzcLCKFACKSKyKVWH/XN8euuRV43RjTD/SJyCWx858DXomtDd0iIjfG7uEUEfcZjUIppZ90lZqrjDEHROSbwIsikgZEgS9iLdS+TER2AP1Y7cpgLUn3UCzZHl8lB6zE/LCIfDd2j1vOYBhKKXTVJqWSjogEjDHeRD+HUmr6tJpaKaWUSjAtGSullFIJpiVjpZRSKsE0GSullFIJpslYKaWUSjBNxkoppVSCaTJWSimlEkyTsVJKKZVg/x/zv0PjgjuN8QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "pd.DataFrame(history.history).plot(figsize=(8, 5))\n",
    "plt.grid(True)\n",
    "plt.xlabel('epoch')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "随着epoch的增加，loss（蓝线）不断下降，val_loss（绿线）先下降后抬升，这种变化趋势说明模型过拟合了。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 测试集评估结果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10000/10000 [==============================] - 0s 42us/sample - loss: 0.3304 - accuracy: 0.8815\n",
      "loss:  0.33044880392551423\n",
      "accuracy:  0.8815\n"
     ]
    }
   ],
   "source": [
    "loss, accuracy = model.evaluate(x_test, y_test)\n",
    "print('loss: ', loss)\n",
    "print('accuracy: ', accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 任务一"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "搭建模型，要求设置两层隐层：\n",
    "\n",
    "- 第一层隐层设置：神经元个数256，初始化方法为glorot_normal，激活函数为tanh\n",
    "\n",
    "\n",
    "- 第二层隐层设置：神经元个数128，初始化方法为glorot_normal，激活函数为tanh\n",
    "\n",
    "然后，运行模型训练和测试集评估代码。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 任务二"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "请尝试使用课上讲的提前终止的方法解决过拟合，使用到的api如下：\n",
    "\n",
    "**`tf.keras.callbacks.EarlyStopping`**\n",
    "\n",
    "用到的参数：\n",
    "\n",
    "- **monitor**：监控的数据，一般为'val_loss'。\n",
    "\n",
    "\n",
    "- **min_delta**：定义模型改善的最小量，只有大于min_delta才会认为模型有改善，默认为0。\n",
    "\n",
    "\n",
    "- **patience**：有多少个epoch，模型没有改善，训练就会停止，默认为0。\n",
    "\n",
    "\n",
    "- **restore_best_weights**：是否使用监控数据最好的模型参数，如果是False，使用的是训练最后一步的模型参数，默认False。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 任务三"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "根据以下表格，设置网络的初始化方法和激活函数，将Test Accuracy的最终结果填入下表。\n",
    "\n",
    "| 权重初始化 | 激活函数 | Test Accuracy |\n",
    "| :--------: | :------: | :-----------: |\n",
    "| glorot_normal | tanh |   0.8815\n",
    "| he_normal    | relu |   0.8853\n",
    "| glorot_normal | elu  |   0.8768\n",
    "| glorot_normal | selu |   0.8789"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "307.125px"
   },
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
